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TITLE 

HAPTIC RENDERING OF VOLUMETRIC SOFT-BODIES OBJECTS 

5 

RELATED APPLICATIONS 

The present application claims the benefits of U.S. Provisional Application 
Serial Number 60/156,852 filed on Sept. 30, 1999. 
1 o FIELD OF THE INVENTION 

The present invention relates to processes and apparatus for interfacing force- 
feedback devices (and like data acquisition transducers) to computers, to add 
touchability to computers. The invention is more particularly associated with such 
15 touchability addition to the art of computer haptic rendering of three-dimensional soft- 
bodied objects. 

BACKGROUND OF THE INVENTION 

20 The interfacing of force-feedback devices (haptic interface devices) to computers 

adds touchability in computer generated interactions, called in the state of the art and herein, 
computer haptics. Two major components of computer haptics are collision detection of 
virtual objects with the haptic interface device, and the determination and display of 
appropriate force feedback to the user via the haptic interface device. Prior art data structures 

25 and algorithms applied to haptic rendering have been adapted from non-pliable surface- 
based (two dimensional) graphic systems. These prior art techniques and systems are 
sometimes inappropriate due to the different characteristics required for haptic rendering of 
three dimensional or volumetric soft bodies. 

It is an object of the present invention to develop software and hardware 

3 0 suitable for haptic rendering of volumetric soft bodies. 


2 


It is a further object of the present invention to provide apparatus and processes 
for providing computer generated haptic renderings that accommodate the needs of 
volumetric soft bodies through recursive processes with efficient management of 
computational resources and viable process and apparatus choices. 
5 A still further object of the present invention is to provide collision detection 

and deformation presentations along with appropriate force feedback to a user to 
provide virtual interactions of a stylus and a volumetric soft body. 

It is still a further object of the present invention to provide a computer haptics 
process and apparatus that operates in real time. 

10 

SUMMARY OF THE INVENTION 

The above objects are satisfied and other limitations of the prior art are 
overcome in the present invention through computer modelling and coding systems 
operating on economically obtainable and usable hardware and operating systems. The 
1 5 modelling/coding systems include : 

1) haptic rendering of 3D volumetric objects using occupancy-map algorithm 
(OMA) for collision detection. [This is accomplished in the present invention as an 
advance over the known OMA used for solid non-deformable convex virtual objects]; 
and 

20 2) chainmail algorithm (CMA) for generation of the real-time feedback forces 

while accommodating deformation of the soft-bodied object. [This is accomplished in 
the present invention as an advance on the known CMA used for calculating the 
behavior of convex surface virtual objects]. 

Minimum and maximums distances in three dimensions that a voxel (a 3D 

25 point with defined x, y, z coordinates relative to a defined vertex/center) can stably 
maintain with neighboring voxels are determined. Relative movement of the position 
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that a voxel takes relative to its center (while maintaining compliance with the minimum 
and maximum distances) is defined as "shear." When these minimums and 
maximums are violated a "collision" occurs, and the direction and relative forces can be 
inferred, so that such forces can be returned via a haptic device to the user. The 
5 minimum and maximum distances and the shears for all the voxels are adjusted to 
conform to the requirements and this calculation is carried out recursively until a stable 
state is evolved for all the voxels. In a preferred embodiment, the stable state is defined 
as a minimum energy state where distances are balanced among the voxels and the 
points are at the center or rest position with no shear. Such a minimum energy state 
1 0 calculation, per se, is well understood in the art. 

An advantage of the present invention is that the characteristics of the voxels 
may be varied to model non-homogeneous soft bodies. 

The physical parameters of the materials involved are known in the art, so that 
the deformation, resistance to forces and penetration and/or bouncing can be modelled, 
15 as is known in the art. 

Using the improved versions of these techniques overcomes the limitations of 
the haptic prior art when applied to volumetric soft-bodied objects. 

Another aspect of the present invention allows for the penetration of a soft 
three-dimensional body while maintaining proper force feedback by use of a proxy 
2 0 form of the penetrating body, a stylus in an embodiment. In yet another aspect of the 
present invention provision is made for the bouncing of one body after colliding with 
another. 

Other objects, features and advantages will be apparent from the following detailed 
description of preferred embodiments taken in conjunction with the accompanying drawings 
25 in which: 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram of the chainmail model for the nearest neighbor distance; 
Figure 2 is a diagram of the chainmail model for shear; 
5 Figure 3 is a representation of the haptic rendering of an object being 

penetrated; 

Figure 4 represents the object of Figure 3 but with the addition of a proxy; and 
Figure 5 is a representation of bounce from a surface when a strong force is 

used. 

10 

DESCRIPTION OF PREFERRED EMBODIMENTS 

Figure 1 illustrates the model used for determining touching or collisions 
between a haptic interface device and a virtual object. This is a first step of haptic 
interaction with virtual objects. In a haptic virtual reality system (HVRS) an interaction 

15 with a virtual object is done by "touching" the object using a haptic interface device - a 
stylus, and feeling the force feedback through the haptic device just as if the object 
were touched in reality. A real-time operation of a HVRS is necessary to be useful (it 
is evident that time delays would make the system useless), and real time operation 
requires efficient collision detection. The present invention provides such efficient 

2 0 collision detection. 

The virtual environment and virtual objects are stored in memory in a three 
dimensional array called an occupancy map. When the occupancy map is initialized, 
each position in the map is set to -1 meaning that position is unoccupied. When a 
virtual object is added to a scene defined by the occupancy map, the appropriate 

25 positions in the occupancy map are set to 0. The borders of the virtual scene follow 
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this same pattern as well. Other such encoding/modelling schemes, known in the art, 
can be also used to advantage. 

The chainmail algorithm operates in a manner similar to the movements of a 
chained armor regarding the individual linked pieces of the armor relative to its 
5 neighbors. With reference to Figure 1, a "voxel" is herein defined a three dimension 
(3D) volumetric "point" 2 which represents one piece of chain mail that is linked to its 
nearest neighbors 4 in three dimensions. It should be noted that a neighbor may be 
along a positive or negative direction on the three axes; therefore there may be up to six 
such nearest neighbors. Figure 1 illustrates the minimum and maximum distances that 

10 a voxel and its linked neighboring voxels are allowed to move maintaining stability. In 
each of the three dimensions there is an minimum 6 and an allowable maximum 8 axis 
location defining an allowed delta position relative to the position of the voxel 2 under 
consideration. Once the occupancy map algorithm, OMA, determines a collision, the 
chainmail algorithm, CMA, determines the geometry of the neighboring vertices 

15 including the direction(s) of the movements. With respect to Figure 2, "shear" herein 
is measured by the positive or negative delta 10 (difference) in position of the voxel 
under consideration compared to its neighbors along each of the three axes. With 
respect to Figures 1 and 2, whenever the minimum 6 or the maximum 8 distances or 
shear distances are violated, the neighboring voxels are adjusted to compensate for the 

2 0 discrepancy so that the entire array of voxels meet the minimum and maximum deltas 
for position and shear. The process is recursively implemented until all the voxels in 
the system reach stability, i.e. meet the minimum, maximum deltas for position and 
shear criteria for the x, y and z directions. 

When a collision occurs the relative positions of the voxels that violate the 

2 5 minimums and maximums of position and shear are used to determine the direction 
and force of the collision. The recursive nature of changing the relative positions of the 
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voxels to meet the minimums and maximums for positions and shear cause the three 
dimensional object to distort in response to the direction and size of the colliding 
voxels, as is known in the art. 

At any given time, the position of the haptic stylus and the occupancy map are 
5 used to determine a collision between the voxel at that position and the stylus tip. 
Furthermore, the direction of approach and "chain mesh"(the actual distances and 
deltas described above for position and shear) of the CMA are taken into account to 
resolve the collision with "ripple effects of other voxels" and the deformation (relative 
movement of the voxel) that occurs due to the collision. 

10 In a preferred embodiment the haptic device is the PHANToM® stylus which 

is well known in the art. In this system the software assumes the stylus is a point that 
interacts with an object as described with the occupancy map. In this system the force 
feedback on the haptic interface device is developed by the computer system as a 
function of the distance from the stylus point to the nearest surface. Modifying the 

15 surface based graphic haptic systems results in at least two major problems that must 
be overcome. One such problem occurs when the stylus approaches through 
penetration of an object. Care must be made to assure that the force which is presented 
by the computer system to the haptic device penetrates an object's surface and 
approaches the opposite surface remains in the proper direction as well as the proper 

2 0 magnitude. Figure 3, illustrates this problem. The stylus 20 collides with the surface 
21. The force 22 is directed by the computer system onto the haptic interface so that 
the user feels the force resisting penetration of the surface. But, as the stylus 
approaches the opposite surface 26, the force reverses direction 24 since the direction 
is computed with reference to surface 26, as if the stylus is penetrating that surface 

2 5 from the opposite direction. Figure 4 illustrates a solution to this problem. Here a 
proxy stylus is created colliding with the surface. The virtual stylus continues to 
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penetrate the object just shown in Figure 3, but the proxy provides a continuous force 
28A, 28B, and 28C that is applied to the haptic device that resists the continued 
penetration of the stylus as the stylus approaches the opposite surface. The second 
problem exists with some volumetric rendering software techniques when the contact is 
5 momentarily lost if the force is strong enough to bounce the point stylus away from the 
surface, although the user is still pushing towards the surface. The net effect is that the 
stylus position and the surface voxel pass through each other losing contact with each 
other. In such a case the voxel relaxes outward towards its original location. This loss 
of contact might result in no force feedback to the haptic device and no pressure 

10 against the surface. Figure 5 illustrates this problem. The user pushes the proxy 
(stylus) 30 with a hard force 31 against the voxel 32 which provides a resisting force 
34 to the haptic device. The surface deforms and bounces away 38 from the proxy and 
there is no resisting force and no force on the voxel 32. The user continues pushing 
and the proxy penetrates beyond the rest position of the voxel 32. With no force on it, 

15 the voxel 32 relaxes back to its rest position 36. At this point the proxy has penetrated 
the surface but the voxel 36 surface is intact - this represents an erroneous operation. 
To help resolve this problem, the collision is checked at the location of the proxy and at 
the six directional points at a fixed radius distance from the proxy until the contact is 
made with a voxel whereupon the proper deformations and forces feedback via the 

2 0 haptic device to the user are established. 

A related problem occurs with the voxel relaxing as described above. The 
virtual computing system must provide a damping action to the relaxing where the 
relaxing incurs only a small amount of vibration. Without this damping the vibration 
will not end and is visually disturbing. 

25 The system software can accommodate non-homogeneous behavior and 

response of the material within the soft bodies. This is accomplished by programming 
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i * 

the force feedback resisting penetration of the bodies by a stylus. The density, 
resilience, viscosity, elasticity, and other physical parameters of the material or materials 
and can be modelled in the software to realistically react to different interactions of 
non-homogeneous materials and the stylus. 
5 As discussed above many computing systems can be used to implement the 

present invention. In a preferred embodiment, a 266 MHz PC running the NT 
operating system, twelve megabytes for representing 11,200 polygons (voxels) was 
implemented. However, other computing systems, such as known in the art, including 
distributed systems and other operating systems can be used to advantage. 

10 While the above description shows utilization of a family of disclosed special 

form OMA-CMA systems of the invention to deal with force-feedback in haptic 
rendering and analysis of soft body objects and conditions thereof, the systems of the 
invention, including the computer modelling and coding techniques thereof, can also be 
applied to other analogous situations of varying feedback intensity of measured or 

15 computed electronic, optical and magnetic responses associated with soft body 
volumes. The present invention incorporates such alternate usages. 

It will now be apparent to those skilled in the art that other embodiments, 
improvements, details, and uses can be made consistent with the letter and spirit of the 
foregoing disclosure and within the scope of this patent, which is limited only by the 

2 0 following claims, construed in accordance with the patent law, including the doctrine of 
equivalents. 

We claim: 
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CLAIMS 

1 . A process for haptic rendering of three-dimensional soft bodied objects for 

virtual interactions comprising the steps of: 
forming a three dimensional occupancy map of voxels, forming the surface and 
5 bound the object, 

forming a multi-dimensional coordinate system for each point, the coordinate 
system defining a vertex center, 

defining minimum and maximums distances of one voxel to all its neighboring 
voxels (occupied neighbor points), 
1 0 defining multi-dimensional maximum offsets that an occupied point can 

maintain relative to its center, 

detecting when the minimums or maximums of distance and/or offsets are 
violated, and in response thereto 

moving the points in violation to locations relative to the neighboring occupied 
15 points and the points' centers that satisfy the minimum and maximum for distance and 
offsets, 

repeating the detecting and moving steps for the entire occupancy map until 
there are essentially no violations. 

2. The process as defined in claim 1 further comprising determining a 
2 0 minimum energy state for all occupied points, and continuing the repeating of the 

detection and moving steps until the entire object is at a minimum energy state. 

3. The process as defined in claim 1, further comprising the steps of 
determining, from the relative moving of the points and the offsets from the center for 
each point, the direction and size of the force of the colliding bodies, and delivering that 

2 5 force in size and direction via a haptic device. 
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4. The process as defined in claim 1, wherein when one body collides with 
and penetrates or bounces from another, further comprising the steps of: 

forming a proxy of the penetrating or bouncing body that maintains its position 
on the surface, and wherein the moving of the points responds to the proxy as well as 
5 the penetrating body. 

5. The process as defined in claim 1 further comprising damping the 
responses of the points as collisions occur. 

6. An apparatus for haptic rendering of three-dimensional soft bodied 
objects for virtual interactions comprising tin combination: 

1 0 means for forming a three dimensional occupancy map of voxels, forming the 

surface and bound the object, 

means for forming a multi-dimensional coordinate system for each point, the 
coordinate system defining a vertex center, 

means for defining minimum and maximums distances of one voxel to all its 
1 5 neighboring voxels (occupied neighbor points), 

means for defining multi-dimensional maximum offsets that an occupied point 
can maintain relative to its center, 

means for detecting when the minimums or maximums of distance and/or 
offsets are violated, and in response thereto 
2 0 means for moving the points in violation to locations relative to the neighboring 

occupied points and the points' centers that satisfy the minimum and maximum for 
distance and offsets, and 

repeating the detecting and moving steps for the entire occupancy map until 
there are no violations. 

25 7. The apparatus as defined in claim 1 further comprising means for 

determining a minimum energy state for all occupied points, and for continuing the 
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repeating of the detection and moving steps until the entire object is at a minimum 
energy state. 

8. The apparatus as defined in claim 1, further comprising the means for 
determining, from the relative moving of the points and the offsets from the center for 

5 each point, the direction and size of the force of the colliding bodies, and for delivering 
that force in size and direction via a haptic device. 

9. The apparatus as defined in claim 1, constructed and arranged so that 
when one body collides with and penetrates or bounces from another, and a proxy of 
the penetrating or bouncing body is formed that maintains its position on the surface, 

1 0 and so that wherein the moving of the points responds to the proxy as well as the 
penetrating body. 

10. The apparatus as defined in claim 1 further comprising means for 
damping the responses of the points as collisions occur. 
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ABSTRACT 

Haptic rendering of three-dimensional soft bodied objects for virtual 
interactions implemented by forming a three dimensional occupancy map of voxels, 
forming the surface of and bounding an object, forming a multi-dimensional coordinate 
5 system, defining minimum and maximums distances of one voxel neighboring voxels 
defining multi-dimensional maximum offsets that an occupied point can 
maintain relative to its center, detecting when the minimums or maximums of distance 
and/or offsets are violated, and in response thereto adjusting to satisfy minimum and 
maximum for distance and offsets, and repeating the detecting and adjustment steps for 
1 0 the entire occupancy map until there are essentially no violations. 
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